Statistical Machine Translation from Arab Vocal Improvisation to Instrumental Melodic Accompaniment

International audienceVocal improvisation is an essential practice in Arab music. The interactivity between the singer and the instru-mentalist(s) is a main feature of this deep-rooted musical form. As part of the interactivity, the instrumentalist re-capitulates, or translates, each vocal sentence upon its completion. In this paper, we present our own parallel corpus of instrumentally accompanied Arab vocal improvisation. The initial size of the corpus is 2779 parallel sentences. We discuss the process of building this corpus as well as the choice of data representation. We also present some statistics about the corpus. Then we present initial experiments on applying statistical machine translation to propose an automatic instrumental accompaniment to Arab vocal improvisation. The results with this small corpus, in comparison to classical machine translation of natural languages, are very promising: a BLEU of 24.62 from Vocal to instrumental and 24.07 from instrumental to vocal

Predicting and Critiquing Machine Virtuosity: Mawwal Accompaniment as Case Study

International audienceThe evaluation of machine virtuosity is critical to improving the quality of virtual instruments, and may also help predict future impact. In this contribution, we evaluate and predict the virtuosity of a statistical machine translation model that provides an automatic responsive accompaniment to mawwal, a genre of Arab vocal improvisation. As an objective evaluation used in natural language processing (BLEU score) did not adequately assess the model's output, we focused on subjective evaluation. First, we culturally locate virtuosity within the particular Arab context of tarab, or modal ecstasy. We then analyze listening test evaluations, which suggest that the corpus size needs to increase to 18K for machine and human accompaniment to be comparable. We also posit that the relationship between quality and inter-evaluator disagreement follows a higher order polynomial function. Finally, we gather suggestions from a musician in a user experience study for improving machine-induced tarab. We were able to infer that the machine's lack of integration into tarab may be due, in part, to its dependence on a tri-gram language model, and instead suggest using a four-or five-gram model. In the conclusion, we note the limitations of language models for music translation

How can machine translation help generate Arab melodic improvisation?

International audienceThis article presents a system to generate Arab music improvisation using machine translation (MT). To reach this goal, we developed a MT model to translate a vocal improvisation into an automatic instrumental oud (Arab lute) response. Given the melodic and non-metric musical form, it was necessary to develop efficient textual representations in order for classical MT models to be as successful as in common NLP applications. We experimented with Statistical and Neural MT to train our parallel corpus (Vocal → Instrument) of 6991 sentences. The best model was then used to generate improvisation by iteratively translating the translations of the most common patterns of each maqām (n-grams), producing elaborated variations conditioned to listener feedback. We constructed a dataset of 717 instrumental improvisations to extract their n-grams. Objective evaluation of MT was conducted at two levels: a sentence-level evaluation using the BLEU metric, and a higher level evaluation using musically informed metrics. Objective measures were consistent with one another. Subjective evaluations by experts from the maqām music tradition were promising, and a useful reference for understanding objective results